Associative memories, also referred to as content addressable memories, are widely used in the fields of pattern matching and identification, expert systems and artificial intelligence. Associative memories have been researched extensively and are being marketed commercially. Unfortunately, however, there is a fundamental scaling problem that may limit the use of associative memories to solve real world problems. In particular, conventional linear associative memories may have inputs that do not associate with each other, which can severely limit the ability of such memories to learn and represent possible nonlinearities, such as interactions among the inputs. Moreover, nonlinear memories may be difficult to scale for real world applications, due to the explosion of weights that is provided between all inputs. This geometric scaling generally is unreasonable to support applications at the scale of complexity that warrants such technology.
The geometric scaling of nonlinear associative memories may also make it increasingly difficult to realize these memories in hardware. In particular, hardware realizations may be more amenable to high speed, parallel processing. However, the number of interconnections that may be needed among hardware elements of nonlinear associative memories may make hardware realizations impractical for real world applications.